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A critical issue in current ‘assessment efforts is the widespread . * 
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use of technically inadequate tests. This study looked at the ttech~ 7 


- nical adequacy of tests chosen by 159 school seehescies ie during a 
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computer simulation of placement decision making. Chi square analysis 
showed * technically adeauiate devices were chosen proportionately more . XQ 3 


often than. techfically {nadequate ‘devices when ee considered - 


(p < .05). ~-This differentiation was not found when ats was 


defined by ven and reLiability (p_ > 05). More éiiphasits ‘on the-' 
importance of ieenateatiy: adequate assessment devices is needed in: 
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* Educational'personnel routinely use tests to gather information 


for’ the purpose of making psychoeduéational decisions about: students, 


The decisions: that are made can have a significant effect on the students' 


~“aife opportunities. When data are collected using tests, it is imperative ‘ 


that the tests used be technically adequate (Salvia‘ & Ysseldyke, 1978; 


Ysseldyke,°*1978, 1979). Ysseldyke (1979) reported that one of the most 
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»critical issues in current assessment. efforts is the widespread use of 
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technically jnadequate test% in decision making. 
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A computer simulated decision-making program was constructed to 


study the extent to which professtonals use technically adequate tests 


in ‘naking placement decisions about students. 
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‘ Development of a Computer Simulated Decision-Making Program 
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,” A computer simulated decision-making program was developed for 
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the purpose of studying the process diagnostic personnel ‘go through in. 


making decisions about potentially handicapped students. Figure 1 i's 


a flow chart illustrating steps in the simulation program, 
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Initially, the prograg pretest collected demographic data on ; 


y. the participants and assessed their knowledge base .in psychoeducat tonal 
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; . °°’ + assessment. Partie@iparnts were then pravided Gogus referral information 
_ . . — P a ae . .# ne * % . Ss 
: a and Kee instructed that- they were to make a pldcement decision for 
: : . . o ° a os. 


Yr. ' , . 
the hypothetical referred g#tudent. ‘eThey were taqld that the computer 
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could provide them with scores and qualitative ‘in¥ormation regarding 
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thé pupil's ‘performance on a qaesers of tests iw seven domains. Parti- 
° bl » . 


‘cipants indicated’ domains in which they wanted information, and then - ' 


.  . selected specific tests on which they wanted scores and/or qualitative 
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ee es were allowed to cont inue selecting domains” and 


‘a specific tests on which they wanted to see information until théy indicated 


: , they wete ready to make a ai gencat decision (or until 29 mfraues had 
me > a i 
yg wiacsen, Upon indicating their dectetone regarding eligibility for 
: : : - ry ; { ° 
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-° services, diagnosis, and prognosis, ‘the participants wére asked a series 


of questions regarding factors that influenced their decision. > é: 
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a One aspect of the simulation program is important for the purposes 
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of this study, specifically, the devices that individuals ,selected during 
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Greseton of gn Ar Archive of Test Data 


As reported above, the sin ulation program. provided. participants \ 
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, with data-en specific tests. All data were within the average fanpe. of 

pep romiance foe a "pupil of the age peteveed: Participants were allowed’ 
free selection of devices to be a yin mee senor making, Tests included 

v | _ Those, that Thurlow: and ° Yaseldyke (1979) | nad. shown were most frequently’ 
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used in making decisions. about learning, disabled ween? 


The list of tests faciuded Bork technically adequate, and technically 
jae 
inadequate devices. Technical adequacy was evaluated on three dimensions: 
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norms, reliability, ahd validity. .First, tests that did not“inclide 
# a8 =e é . 
“this information in their.matuals were judged technically inadequ e. 
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The investigators did. not go+beyond manuals in’search of. research on . ) 
the tests; we believe test, authors must report the data in their man- 
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uals. Second, criteria specified by Salvia and Ysseldyke’ (1978), by 
 Ysstldyke (1978), sand in the APA Standards document. (1972) were used ar 
to evaluate the technical adequacy of the tésts.. 
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,The devices available for selection a gas simulated diagnostic ee 


session and‘their technical adequacy relative ta norms, reliability, and co™ 7 
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validity are listed im Table 1. Twenty-four percent (i.e., 12 of a2) 
of the detieea ack rated as having technical ty dieqiae norms and < ; ea 
sae: thirty-three Sega were rated as having sidequate veltabiitucys 
: = 8 . ‘ us 
sixty-five percent were ratéd as having inadequate norms: ‘fifty-nirie 
‘pencent sa avin inadequate reliability; and siniyssaven- percent as 
3 wa ; \ 


- having i aan validity. ae / 
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Insert’ Table 1 about here 4 ae a 
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Subjett's oa | i : : ss ’ 
‘ Subjects were 159 professionals Ton public and private schools | 

in Minnesota. All participants were volunteers who had previously — > 

participated in at least two placement team meetings.° Disciplines re- : 


presented within the sample include regular education teachers (N = 55), 


- , . 
special education.teachers (N= 47),-administrators (N = 16), school ' 

. 7 . 
psychologists (N = 25), and support. personnel (e.g., social worker, ’ 
nurse, etc.) (N ="16). | ; | ‘ 
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" . Procedures 
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; Data were collected using a Telray remote computer terminal 
. L J . . . 


a“ : . ; i , . . 
and were accessed by the Cybernet network. All data were collected 


in the professionals! home school districts. ‘Each subject participated 
; ee 


in’ the interactive program*£er approximately 45 minutes. 
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: . . Results— : 
# , The extent to which technically adequate or inadequate devices - 7 8 


were selected during the diagnostic simulation may be “derived from 
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7 a ‘Table 2; presegted is the total number of times the devices were . 
‘ : oo : ; 
; ; eo 7 
" selected. a 
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ra . Insert Table. 2 about. here 
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a * “* Analyses of the results of three separate Chi square tests ,sug- 
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, gested that individuals tended to select devices with similar technical 
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. ° characteristics for reliability and'validity (e.g-,; ri < 1.0) but that + 
is > ° , : 
. their choices with regard to the technical quality of the tests' norms 
: were differentially distributed Ge = 46.43 p < 05). Specifically, 
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subjécts selected more test devices * technically adequate norms _~ i 


and fewer devices with technically ina ate norms than might be ex- 
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pected by chance. a. y 
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That avekeentendve who engage {rf assessment of children. should 


do so ‘with technicglly adequate devices seems an obvious requirement 


o ; and/or recommended practice. Salvia and Ysseldyke (1978) have point’ed 
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out'that a nunber{ of the currently popular agsessment devices used 
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- by special educators are bechapcatty snadcquate Dased on -profeasional 
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standards for best practices (APA, 1972). No controls exigt through 


which the pibication of tests with inadequate norms, reliability, an4/or . 

4 . 4 
validity may be monitored; the burden of appropriate Seleseian and use . 
then resfs with the professional who enuases in, psychoeducational : 
asceseusars ” 7 7 
"” The extent to which professionals (1.e., school, psychologists, ae 
special edscatien, tc ere, etc.) selected technically adequate. or are. 
Weavequate devices during a diagnostic. gnuisoaes didvessed by r 
this research. . An analysis of the-results aupeesced tit school per- = 
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sonnel tended to select, devices with’ tgchnically adequate ‘reliability 
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and validity ce as they selecte devices abese relia- 
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pabety ane validity. With regard to test norms, howé@ver, a 2 
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_ tended to sélect de ices judged to “have adequate norms more often than 


they seleoted inad uately normed, devices. ° ? | 
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- The participants, in *this study were all ,individuals who had 0 


already participated in ‘nalctag placement decisions. We believe it is 


imperative that ineré pane attention be given in both inservice and 


preservice training to the importance of technical sdiequacy in the 
é * . < - _4 : ! 
selectiian of instruments, fot use-in dec?sion. making. te \ 
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Figure 1. Flow Chart illustrating the Diagnostic Simulation Program. 
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